Most of the existing digit recognition algorithms recognize single-type digits, and can not recognize multi-source digits. Aiming at the character recognition scenarios with handwritten digits and digital tube digits, a multi-source digit recognition algorithm based on improved Convolutional Neural Network (CNN) was proposed. Firstly, a mixed data set consisting of handwritten and digital tube digits was established by using the samples collected from the field of digital display instrument manufacturer and MINIST data set. Then, considering better robustness, an improved CNN was proposed, which was trained by the above mixed data set, and a network was realized to recognize multi-type digits. Finally, the trained neural network model was successfully applied to the multi-source digit recognition scene of RoboMaster robotics competition. The test results show that, the overall recognition accuracy of the proposed algorithm is stable and high, and it has good robustness and generalization ability.